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Abstract 

We find a generating function expressed as a continued fraction that enumerates 
ordered trees by the number of vertices at different levels. Several Catalan problems 
are mapped to an ordered-tree problem and their generating functions also expressed 
as a continued fraction. Among these problems is the enumeration of (132)-pattern 
avoiding permutations that have a given number of increasing patterns of length k. 
This extends and illuminates a result of Robertson, Wilf and Zeilberger for the case 
k = 3. 



1 Introduction 

A Catalan problem is any enumerative problem that produces the Catalan sequence of 
numbers or one of its many g-analogs. Stanley 0] provides a catalog of 66 Catalan problems. 
Interestingly, many of the generating functions that arise from these problems can be given 
as a continued fraction with a simple yet elegant form. Two of these generating functions 
are reproduced below and a third we derive anew. Our intent is to show that the first two 
continued fractions are special instances of the third with the implication that many others 
are as well. We begin with the three Catalan problems and their corresponding generating 
functions. 

Problem 1. A (132) pattern (respectively, a (123) pattern) in a permutation it of length 
n is a triple l<i<j<k<nof indices for which 7r(z) < 7r(/c) < 7r(j) (respectively, 
ir(i) < ir(j) < Tr{k)). Let f r {n) denote the number of permutations 7r of length n that have 
no (132) patterns and exactly r (123) patterns. Recently, Robertson, Wilf and Zeilberger 
derived the generating function 
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in which the Ith. numerator is zq\ 2 > (I is for level in anticipation of Problem 3). 

Problem 2. The number of lattice paths from (0, 0) to (n, n) with steps (1, 0) and (0, 1) 
that never rise above the line y = x is a Catalan number. Let P be such a path, A(P) the 
area under the path (and above the x-axis), and let C n (q) = Y.pQ^^- Then a generating 
function is given by (see Exercise 6.34 in Stanley and replace the x therein with zq ) 
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in which the Zth numerator is zq 1 . 

Problem 3. The number of ordered trees (also known as plane trees) on n edges is a 
Catalan number. The level of a vertex is the number of edges on the unique path from 
the root to the vertex. Thus, the root is the unique vertex at level zero and the vertices at 
level one are adjacent to the root. Let TJ li ^ 2) ... be the number of ordered trees that have Ik 
vertices at level k > and let Vk, k > 0, be indeterminates. The generating function T that 
enumerates ordered trees by the number of vertices at each level is defined as 

T( Vl ,v 2 ,...)= e T h,h,.vH 2 ---- 

h,l 2 ,:.>0 

The first few terms (number of edges n < 3) of T are 

T{yx, v 2 , ■ ■ ■) = 1 + Vi + viv 2 + vf + Vxv 2 v 3 + 2vfv 2 + v x v\ + v\ H . 

That T can be written as a continued fraction that subsumes the previous continued fractions 
is our main result. It is simple, yet has some interesting applications. 
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Theorem 1. The generating function that enumerates ordered trees by the number of ver- 
tices at each level is 

T(v u v 2 , ■■■) = 

1 V -l 
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Proof. We exploit the natural recursive property of ordered trees to obtain a recursion for 
T. The recursion immediately leads to the continued fraction. Any ordered tree on more 
than one vertex can be constructed from a collection of others (the subtrees) by joining the 
roots of these subtrees to a new vertex. The new vertex becomes the root of the tree under 
construction. Note that the level of a vertex in a subtree increases by one after the new root 
is inserted. The function T(vi, v 2 , . . .) enumerates the choices for a subtree and each of these 
choices contributes a factor of ViT(v 2 ,v 3 , . . .) because of the level changes. The factor of vi 
is present because the root of the subtree becomes a vertex at level one. Thus, the trees 
with k subtrees (of the root) are enumerated by v^T k (v 2 ,v 3 , . . .). The generating function 
satisfies 

T(v u v 2 ,...) = l + v 1 T(v 2 ,v 3 ,...)+v 2 1 T 2 (v 2 ,v 3 ,...) + ... 

1 



1 - V X T(v 2 ,V 3 , ...)' 

Iteration of the last functional recursion produces the continued fraction. □ 

An immediate application is obtained by replacing each indeterminate Vk with z denoted 
simply as T(z). The resulting function enumerates ordered trees by the number of edges and 
is 

T(z) = — 



1 - 



z 
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The well-known solution of the above generates the Catalan numbers and is T(z) = (1 — 
VI - Az)/2z. 
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The more challenging applications are the evaluations needed to produce the continued 
fractions of permutations (Equation [I] of Problem 1) and lattice paths (Equation ^ of Problem 
2). Both applications require that we map their respective problems to an ordered-tree 
problem. These mappings are of interest in their own right and we explore them now. We 
begin with the lattice path problem because it is simpler and the mapping is already known. 

2 Lattice paths and ordered trees 

We draw our ordered trees with the root at the top and proceed downward to the leaves. 
The first leaf is the leftmost leaf in the drawing and the remaining leaves are referred to 
by their positions in a left-to-right order. A preorder (depth-first) traversal of the ordered 
tree provides a well-known correspondence with a lattice path. When an edge of the tree is 
traversed downward away from the root we take a (1, 0) (east) step in the lattice, otherwise 
we take a (0, 1) (north) step. In this manner, an ordered tree with n edges corresponds to a 
unique lattice path from (0, 0) to (n,n). 

If the path P corresponds to the tree T, then we need to determine what statistic of the 
tree corresponds to the area A(P) under the path. We let A(T) = A(P) be this statistic of 
the tree and claim that it depends only on the vertex levels. 

Lemma 2. IfT is an ordered tree on n edges, then 



where level (f) is the level of vertex v. 

Proof. Let w be the rightmost leaf of the tree T. Our immediate interest is to calculate the 
area under the east step that arises in the lattice path by traversing the last edge to this 
leaf downward away from the root. This area is equal to the height that the east step has 
in the lattice path and is equal to the number of north steps that have occurred prior to the 
east step. There are a total of n north steps in the lattice path (one for each edge of the 
tree) and level(tu) remaining north steps after the east step. Hence, the east step is at height 
n — level (w) in the lattice. If we now delete the leaf w from the tree T, then the resulting tree 
has a lattice path with area n — level (w) less than that of T. A formal inductive argument 
on the number of edges provides the result. □ 

We use the lemma to prove the following continued-fraction result. The result is the same 
as that given in Equation ||. 




vertices v 
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Theorem 3. IfC n (q) = Y,tQ A ^ enumerates the set of ordered trees on n edges by the area 
under their corresponding lattice paths, then 
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in which the Ith numerator is zq 1 . 

Proof. Let T be an ordered tree on n edges and assign to every nonroot vertex at level I > 
the value zq 1 . The product of all these values is then z n q^^ level( - v \ Summing over all ordered 
trees on n edges we have by the lemma 

^£ g E^vel(,) = ,n£ g CT)-^) = ,n g Cr)C n (l/g). 
T T 

The sum of these over all n > then enumerates ordered trees and the generating function 
is given by the continued fraction of Theorem [I] with vi = zq 1 . □ 



3 Permutations and ordered trees 

The previous problem used an existing bijection between the set of ordered trees and the set 
of lattice paths to get the desired result. We seek a similar approach for ordered trees and 
permutations. There are many ways to map a permutation onto a tree (often an unordered 
tree) but none of these serve our needs. The mapping we introduce appears to be new. 

Let T be an ordered tree on n edges. We use a preorder traversal of T to label the 
nonroot vertices in decreasing order with the integers n, n — 1, . . . , 1. Thus, the first vertex 
visited gets the label n and the last receives 1. We now construct a permutation written 
as a word by reading the labeled tree in postorder. We again traverse the tree from left to 
right and record the label of a vertex when we last visit it. In Catalan fashion, the five 
ordered trees and their corresponding permutations are shown in Figure |l|. Note that the 
only permutation missing from those of length three is 132. The (132) pattern has been 
avoided. Also note that there is exactly one permutation with a (123) pattern (the first 
permutation shown). Thus, recalling the definition of f r (n) given in the introduction we 
have /o(3) = 4, /i(3) = 1, and / r (3) = 0,r > 1. We generalize these observations after 
introducing some useful notation. 
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123 231 312 213 321 




Figure 1: The five ordered trees on 3 edges and their permutations. 



If T is an ordered tree on n edges, then we let vr(T) be its corresponding permutation 
written as a word on the numbers 1,2, ... ,n. We let 7r(T, k) be the same permutation except 
we use the corresponding numbers 1 + k, 2 + k, . . . , n + k. For example, the first tree shown 
in the figure has tt(T) = 7r(T, 0) = 123 and 7r(T, 1) = 234. For emphasis, we denote the 
concatenation of two words it and n' as ttAtt' (instead of the usual mr'). The following lemma 
describes how the permutation of a tree can be constructed from those of its subtrees. 

Lemma 4. Let T be an ordered tree on n edges with subtrees T±, T2, ■ ■ ■ , T s on n\, 712, ■ ■ ■ , n s 
edges, respectively. Let Nq = n and Nf. = Nk-i — Uk — 1, k = 1, 2, . . . , s, then 

tt(T) = 7r(T x , iVi) A N A 7r(T 2 , N 2 ) A N x A . . . A vr(T s , N s ) A N s ^. 

Proof. Note that N = n is the total number of vertices to be labeled, that Ni is the total 
number of vertices to be labeled after those of the subtree T\, and so on. Since the vertices of 
T are labeled in decreasing order using a preorder traversal, i < j implies that all vertices of 
Tj receive labels greater than those of Tj. Since vr(T) is constructed by reading these labels 
in postorder, i < j implies that the vertex labels in Tj appear in n(T) before any of those 
in Tj. Thus, vr(T) begins as vr(Ti) with each number incremented by N\, i.e., as ir(Ti,Ni). 
The root of this subtree is the first vertex visited in the preorder traversal and receives the 
label n = N . It is read last among the vertices of T\ in postorder, however, so that tt(T) 
begins as n(Ti, Ni) A Nq. The general case follows similarly. □ 

Note that the lemma also provides the means to prove that the mapping T — > ir(T) is 
injective. That it is onto the set of all (132)-pattern avoiding permutations is proved in the 
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next theorem. Before proceeding to this theorem we present another lemma which enables 
us to count (123) patterns and their generalization. An increasing pattern of length k, k > 0, 
in a permutation tt of length n is a fc-tuple 1 < i\ < i 2 < • • • < ik < n of indices for which 
tt(zi) < ix(i 2 ) < ... < 7r(4). 

Lemma 5. Let T be an ordered tree on n edges and Vk a subset of k vertices, < k < n, 
none of which are the root, then the labels of these vertices provide an increasing pattern of 
length k in 7r(T) if and only if they lie along a path from the root to some leaf. 

Proof. We induct on n. If n = 1, then k = 1 and the lemma is obvious. In fact, this is the 
case for all n whenever k — 1. We assume that the lemma is true for any ordered tree on n 
or fewer edges, n > 0, and let T be a tree on n + 1 edges. Suppose that 7r(T) contains an 
increasing pattern of length k > 1. We let V{ be the vertex in T that provides the i in such a 
pattern, i = 1, 2, . . . , k. Then Vk must receive a larger label than V\ implies that the subtree 
containing Vk precedes or is the same as that of v\ (see the proof of Lemma (|). However, 
the label of Vk must be read after that of v\ implies that its subtree must follow or be the 
same as that of V\. The conclusion is that they are in the same subtree. A similar argument 
applies to t>j and V\, % > 1, so that all the vertices must be in the same subtree, say T;. Thus, 
the increasing pattern of length k lies entirely within 7r(Tj, iVj) A A^_i. 

Recall that it is the root of the subtree that receives the label iVj_i. If the root of the 
subtree is one of the vertices providing the increasing pattern, then it must be Vk- We must 
consider two cases depending on whether the subtree root is v k or not. 

If it is not, then the pattern lies entirely within 7r(Tj, iVj) and corresponds uniquely to an 
increasing pattern of length k in 7r(Tj). By the inductive hypothesis, the vertices Vi,v 2 , . . . ,Vk 
must lie along some path from the root of the subtree to a leaf. Necessarily, this is also a 
path from the root of T to a leaf as required by this lemma. 

If the root of the subtree is Vk, then v±,v 2 , ■ ■ ■ ,Vk-i provide the labels for an increasing 
pattern of length k — 1 in 7r(Tj, iVj). This pattern corresponds to a unique increasing pattern 
of length k — 1 in vr(Tj) and again by hypothesis the vertices v%, v 2 , ■ ■ ■ , v k-i must lie along a 
path from the root of the subtree to some leaf of the subtree. This path together with the 
root of the subtree provides the needed path of this lemma. 

Conversely, if Vk = {vi,v 2 , ■ ■ ■ ,Vk} is a subset of nonroot vertices of T, where we may 
assume the label of Vi is less than that of Vj whenever i < j, and the vertices lie along a path 
from the root to a leaf, then lies below Vj on this path whenever i < j. Thus, when the 
labels are read in postorder, the label of Vi is read prior to that of Vj, i < j. The vertices 
then provide an increasing pattern of length k in 7r(T). 

□ 

The two previous lemmas enable us to prove the following interesting combinatorial 
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theorem. Its corollary establishes a continued fraction as the generating function of (132)- 
avoiding permutations by number of increasing patterns of length k. 

Theorem 6. A permutation tt avoids the (132) pattern if and only if n = 7r(T) for some 
tree T. If this is the case, then the number of increasing patterns of length k depends only 
on the levels of the vertices in the tree and is given by J2 V (^t-i^ 1 ) • 

Proof. Suppose that tt(T) contains a (132) pattern and that T is among the smallest such 
trees. We let t>3 be a vertex in T that provides the 3 in such a pattern and let t>i,t>2 be the 
vertices that provide the corresponding 1 and 2, respectively. Then V3 must receive a larger 
label than V\ implies that the subtree containing v 3 precedes or is the same as that of v\ (see 
the proof of Lemma |]). However, the label of v$ must be read after that of V\ implies that 
its subtree must follow or be the same as that of V\. The conclusion is that they are in the 
same subtree. A similar argument applies to v 2 and v\ so that all three vertices must be in 
the same subtree, say Tj. 

Also note that none of them can be the root of the subtree since the root receives the 
largest label among the vertices of the tree and, hence, can not be V\ or t> 2 . Its label appears 
later in ir(T) than the others in the subtree implies that the root can not be V3. Thus, 
the (132) pattern lies entirely within 7r(Tj, Ni) which implies that 7r(T;) itself must have a 
(132) pattern contradicting our choice of T. Since the number of (132)-pattern avoiding 
permutations of length n and the number of ordered trees on n edges are the same Catalan 
number, the mapping T — > vr(T) is a bijection between these sets. It is an instructive exercise 
to construct T from a (132)-pattern avoiding permutation. 

It remains to determine the number of increasing patterns of length k in 7r(T). As a 
result of Lemma |5] it is only necessary to count the number of vertex subsets of size k, none 
of which are the root, such that the vertices lie along a path from the root to a leaf. We 
claim this number is J2v ( lev t-i _1 ) as stated. To see this, let v be any vertex of T and choose 
Vk = v. There are level (v) — 1 nonroot vertices other than v along the unique path from the 
root to v. From these we select any k — 1 of them, which together with v, form the required 
subset. It is clear that every subset with the required properties arises this way and we are 
done. 

□ 

We now use the theorem to write a generating function as a continued fraction. We 
let f^ k \n) denote the number of permutations of length n that have no (132) pattern and 
exactly r increasing patterns of length k. The case k = 3 is that considered by Robertson, 
Wilf and Zeilberger QTJ . 
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Corollary 7. The generating function that enumerates (132) -pattern avoiding permutations 
of length n by number of increasing patterns of length k is 

E ti k \n)z n q r = (3) 

r,n>0 1 jV l 

No 
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in which the Ith numerator Ni is zq^ k 

Proof. Let T be an ordered tree on n edges and assign to every nonroot vertex at level / > 

the value zq^- 1 ' . The product of all these values is then z n q^^ \ k ~ l > and the result 
follows from the previous theorem and Theorem [I]. □ 
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